
 <!DOCTYPE HTML>
<html>
<head><meta name="generator" content="Hexo 3.9.0">
  <meta charset="UTF-8">
  
    <title>elasticsearch template及index的优化参数 | Zong&#39;s blog</title>
    <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=3, minimum-scale=1">
    
    <meta name="author" content="Zong">
    
    <meta name="description" content="elasticsearch的template是很有用，它描述了index的mapping（类似表结构）、setting参数优化，特别是用日期分割index的情况，提前建好template能在自动创建index的时候套用上。一般日志类、自动创建多个同类index等情况推荐在创建index前先定义好。t">
    
    
    
    
    
    <link rel="icon" href="/img/favicon.ico">
    
    
    <link rel="apple-touch-icon" href="/img/pacman.jpg">
    <link rel="apple-touch-icon-precomposed" href="/img/pacman.jpg">
    
    <link rel="stylesheet" href="/css/style.css">
</head>
</html>
  <body>
    <header>
      <div>
		
			<div id="imglogo">
				<a href="/"><img src="/img/logo.svg" alt="Zong&#39;s blog" title="Zong&#39;s blog"/></a>
			</div>
			
			<div id="textlogo">
				<h1 class="site-name"><a href="/" title="Zong&#39;s blog">Zong&#39;s blog</a></h1>
				<h2 class="blog-motto">日常积累，技术分享</h2>
			</div>
			<div class="navbar"><a class="navbutton navmobile" href="#" title="Menu">
			</a></div>
			<nav class="animated">
				<ul>
					<ul>
					 
						<li><a href="/">Home</a></li>
					
						<li><a href="/archives">Archives</a></li>
					
						<li><a href="/categories/运维">运维</a></li>
					
						<li><a href="/categories/容器架构">容器架构</a></li>
					
					<li>
					
					<form class="search" action="//baidu.com/s" method="get" accept-charset="utf-8">
						<label>Search</label>
						<input type="text" id="search" name="wd" autocomplete="off" maxlength="20" placeholder="Search" />
                        <input name=tn type=hidden value="bds">
                        <input name=cl type=hidden value="3">
                        <input name=ct type=hidden value="2097152">
						<input type="hidden" name="si" value="www.lstop.pub">
					</form>
					
					</li>
				</ul>
			</nav>			
</div>

    </header>
    <div id="container">
      <div id="main" class="post" itemscope itemprop="blogPost">
	<article itemprop="articleBody"> 
		<header class="article-info clearfix">
  <h1 itemprop="name">
    
      <a href="/2020/09/04/elasticsearch-template及index的优化参数/" title="elasticsearch template及index的优化参数" itemprop="url">elasticsearch template及index的优化参数</a>
  </h1>
  <p class="article-author">By
    
      <a href="http://www.lstop.pub" title="Zong">Zong</a>
    </p>
  <p class="article-time">
    <time datetime="2020-09-04T10:12:39.000Z" itemprop="datePublished">2020-09-04</time>
    
  </p>
</header>

	<div class="article-content">
		
		
		<div id="toc" class="toc-article">
			<strong class="toc-title">Contents</strong>
		
		</div>
		
		<p>elasticsearch的template是很有用，它描述了index的mapping（类似表结构）、setting参数优化，特别是用日期分割index的情况，提前建好template能在自动创建index的时候套用上。一般日志类、自动创建多个同类index等情况推荐在创建index前先定义好。<br>template可以创建多个，一个template可以对应一组相同结构的index，其定义大概如下：</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line">PUT _template/filebeat-bxr-app-pro</span><br><span class="line">&#123;</span><br><span class="line">  &quot;index_patterns&quot;: [&quot;filebeat-bxr-app-pro-*&quot;],</span><br><span class="line">  &quot;settings&quot;: &#123;</span><br><span class="line">    &quot;number_of_shards&quot;: 8,</span><br><span class="line">	&quot;number_of_replicas&quot;: 1,</span><br><span class="line">    &quot;refresh_interval&quot;: &quot;120s&quot;,</span><br><span class="line">    &quot;index.merge.scheduler.max_thread_count&quot;: 1,</span><br><span class="line">    &quot;index.translog.durability&quot;: &quot;async&quot;,</span><br><span class="line">    &quot;index.translog.sync_interval&quot;: &quot;60s&quot;,</span><br><span class="line">	&quot;index.translog.flush_threshold_size&quot;: &quot;1g&quot;</span><br><span class="line">  &#125;,</span><br><span class="line">  &quot;mappings&quot;: &#123;</span><br><span class="line">    &quot;doc&quot;: &#123;</span><br><span class="line">      &quot;properties&quot;: &#123;</span><br><span class="line">        &quot;level&quot;: &#123;</span><br><span class="line">          &quot;type&quot; : &quot;keyword&quot;</span><br><span class="line">        &#125;,</span><br><span class="line">        &quot;appname&quot;: &#123;</span><br><span class="line">          &quot;type&quot; : &quot;keyword&quot;</span><br><span class="line">        &#125;,</span><br><span class="line">        &quot;podname&quot;: &#123;</span><br><span class="line">          &quot;type&quot; : &quot;keyword&quot;</span><br><span class="line">        &#125;</span><br><span class="line">      &#125;</span><br><span class="line">    &#125;</span><br><span class="line">  &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>

<blockquote>
<p>index_patterns: index的名称，可以用*表示多个index</p>
<p>settings: index的参数，包括定义shard、replica等</p>
<ul>
<li>number_of_replicas: 数据冗余度，1表示1个冗余副本</li>
<li>number_of_shards:   分片数，一个index分开多个分片，保存在不同的集群节点，配合routing参数可以更快进行查询操作，类似mysql的分表</li>
<li>refresh_interval:   刷新间隔，在 Elasticsearch 中，写入和打开一个新段的轻量的过程叫做 refresh。默认情况下每个分片会每秒自动刷新一次。这就是为什么我们说 Elasticsearch 是近实时搜索: 文档的变化并不是立即对搜索可见，但会在一秒之内变为可见。如果写入数据量大，调整这个参数可以有效降低io压力。</li>
<li>merge.scheduler.max_thread_count:  索引 merge 最大线程数，该参数可以有效调节写入的性能。因为在存储介质上并发写，由于寻址的原因，写入性能不会提升，只会降低。</li>
<li>translog: 每次 index、bulk、delete、update 完成的时候，一定会触发刷新 translog 到磁盘上，类似mysql binlog</li>
<li>translog.durability:            async异步写入translog</li>
<li>translog.flush_threshold_size:  log文件大小</li>
</ul>
<p>mappings: index中关键字的类型定义，类似mysql的字段属性，也可以设置分词器</p>
<ul>
<li>analyzer： 分词器，默认是英文，也有中文分词器</li>
<li>copy_to： 将该字段复制到目标字段，实现类似_all的作用，不会出现在_source中，只用来搜索。列如把字段first_name，last_name，合成full_name做搜索。</li>
<li>index： 控制当前字段是否索引，默认为true，即记录索引，false不记录，即不可搜索</li>
</ul>
<p>更多请参考:<br><a href="https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html" target="_blank" rel="noopener">官方文档</a><br><a href="https://juejin.im/post/6844903661722664973#heading-14" target="_blank" rel="noopener">Elasticsearch 6.x Mapping设置</a></p>
</blockquote>
  
	</div>
		<footer class="article-footer clearfix">

  <div class="article-tags">
  
  <span></span> <a href="/tags/elasticsearch/">elasticsearch</a>
  </div>


<div class="article-categories">
  <span></span>
  <a class="article-category-link" href="/categories/运维/">运维</a>
</div>



<div class="article-share" id="share">

  <div data-url="http://www.lstop.pub/2020/09/04/elasticsearch-template及index的优化参数/" data-title="elasticsearch template及index的优化参数 | Zong&#39;s blog" data-tsina="" class="share clearfix">
  </div>

</div>
</footer>   	       
	</article>
	
<nav class="article-nav clearfix">
 
 <div class="prev" >
 <a href="/2020/09/18/elasticsearch-reindex/" title="elasticsearch reindex">
  <strong>PREVIOUS:</strong><br/>
  <span>
  elasticsearch reindex</span>
</a>
</div>


<div class="next">
<a href="/2020/08/28/elasticsearch备份到hdfs/"  title="elasticsearch备份到hdfs">
 <strong>NEXT:</strong><br/> 
 <span>elasticsearch备份到hdfs
</span>
</a>
</div>

</nav>

	
</div>  
      <div class="openaside"><a class="navbutton" href="#" title="Show Sidebar"></a></div>

  <div id="toc" class="toc-aside">
  <strong class="toc-title">Contents</strong>
  
  </div>

<div id="asidepart">
<div class="closeaside"><a class="closebutton" href="#" title="Hide Sidebar"></a></div>
<aside class="clearfix">

  
<div class="tagslist">
	<p class="asidetitle">Tags</p>
		<ul class="clearfix">
		
			<li><a href="/tags/Airtest/" title="Airtest">Airtest<sup>1</sup></a></li>
		
			<li><a href="/tags/DNS/" title="DNS">DNS<sup>1</sup></a></li>
		
			<li><a href="/tags/GitLab/" title="GitLab">GitLab<sup>1</sup></a></li>
		
			<li><a href="/tags/K8s/" title="K8s">K8s<sup>8</sup></a></li>
		
			<li><a href="/tags/Linux/" title="Linux">Linux<sup>1</sup></a></li>
		
			<li><a href="/tags/MongoDB/" title="MongoDB">MongoDB<sup>2</sup></a></li>
		
			<li><a href="/tags/OpenWrt/" title="OpenWrt">OpenWrt<sup>1</sup></a></li>
		
			<li><a href="/tags/Python/" title="Python">Python<sup>2</sup></a></li>
		
			<li><a href="/tags/RabbitMQ/" title="RabbitMQ">RabbitMQ<sup>1</sup></a></li>
		
			<li><a href="/tags/calico/" title="calico">calico<sup>1</sup></a></li>
		
			<li><a href="/tags/cdn/" title="cdn">cdn<sup>1</sup></a></li>
		
			<li><a href="/tags/docker/" title="docker">docker<sup>3</sup></a></li>
		
			<li><a href="/tags/docker-registry/" title="docker registry">docker registry<sup>1</sup></a></li>
		
			<li><a href="/tags/elasticsearch/" title="elasticsearch">elasticsearch<sup>3</sup></a></li>
		
			<li><a href="/tags/elk/" title="elk">elk<sup>3</sup></a></li>
		
			<li><a href="/tags/k8s/" title="k8s">k8s<sup>3</sup></a></li>
		
			<li><a href="/tags/kubernetes/" title="kubernetes">kubernetes<sup>1</sup></a></li>
		
			<li><a href="/tags/nginx/" title="nginx">nginx<sup>1</sup></a></li>
		
			<li><a href="/tags/python/" title="python">python<sup>1</sup></a></li>
		
			<li><a href="/tags/tomcat/" title="tomcat">tomcat<sup>1</sup></a></li>
		
		</ul>
</div>


  <div class="linkslist">
  <p class="asidetitle">Links</p>
    <ul>
      <li><a href="http://www.v2ex.com/?r=zong400" target="_blank" title="V2EX">V2EX</a></li>
      <li><a href="http://hexo.io" target="_blank" title="Hexo">Hexo</a></li>
	  <li><a href="https://promotion.aliyun.com/ntms/yunparter/invite.html?userCode=s0bh6uzq" target="_blank" title="阿里云">阿里云</a></li>
	  <li><a href="https://cloud.tencent.com/redirect.php?redirect=1014&cps_key=5bd9deb84d4d9f34b65fb934e12d03e3&from=console" target="_blank" title="腾讯云">腾讯云</a></li>
    </ul>
</div>


</aside>
</div>
    </div>
    <footer><div id="footer" >
	
	
	<div class="social-font" class="clearfix">
		
		
		
		
	</div>
		<p class="copyright">Hosted by <a href="https://pages.coding.me/" target="_blank" title="Coding Pages">Coding Pages</a></p>
		<p class="copyright">Powered by <a href="http://hexo.io" target="_blank" title="hexo">hexo</a> and Theme by <a href="https://github.com/wizicer/iceman" target="_blank" title="Iceman">Iceman</a> © 2020 
		
		<a href="http://www.lstop.pub" target="_blank" title="Zong">Zong</a>
		
		</p>
</div>
</footer>
    <script src="//cdn.staticfile.org/jquery/2.1.0/jquery.min.js"></script>
<script type="text/javascript">
$(document).ready(function(){ 
  $('.navbar').click(function(){
    $('header nav').toggleClass('shownav');
  });
  var myWidth = 0;
  function getSize(){
    if( typeof( window.innerWidth ) == 'number' ) {
      myWidth = window.innerWidth;
    } else if( document.documentElement && document.documentElement.clientWidth) {
      myWidth = document.documentElement.clientWidth;
    };
  };
  var m = $('#main'),
      a = $('#asidepart'),
      c = $('.closeaside'),
      o = $('.openaside');
  $(window).resize(function(){
    getSize(); 
    if (myWidth >= 1024) {
      $('header nav').removeClass('shownav');
    }else
    {
      m.removeClass('moveMain');
      a.css('display', 'block').removeClass('fadeOut');
      o.css('display', 'none');
      
      $('#toc.toc-aside').css('display', 'none');
        
    }
  });
  c.click(function(){
    a.addClass('fadeOut').css('display', 'none');
    o.css('display', 'block').addClass('fadeIn');
    m.addClass('moveMain');
  });
  o.click(function(){
    o.css('display', 'none').removeClass('beforeFadeIn');
    a.css('display', 'block').removeClass('fadeOut').addClass('fadeIn');      
    m.removeClass('moveMain');
  });
  $(window).scroll(function(){
    o.css("top",Math.max(80,260-$(this).scrollTop()));
  });
});
</script>

<script type="text/javascript">
$(document).ready(function(){ 
  var ai = $('.article-content>iframe'),
      ae = $('.article-content>embed'),
      t  = $('#toc'),
      h  = $('article h2')
      ah = $('article h2'),
      ta = $('#toc.toc-aside'),
      o  = $('.openaside'),
      c  = $('.closeaside');
  if(ai.length>0){
    ai.wrap('<div class="video-container" />');
  };
  if(ae.length>0){
   ae.wrap('<div class="video-container" />');
  };
  if(ah.length==0){
    t.css('display','none');
  }else{
    c.click(function(){
      ta.css('display', 'block').addClass('fadeIn');
    });
    o.click(function(){
      ta.css('display', 'none');
    });
    $(window).scroll(function(){
      ta.css("top",Math.max(140,320-$(this).scrollTop()));
    });
  };
});
</script>


<script type="text/javascript">
$(document).ready(function(){ 
  var $this = $('.share'),
      url = $this.attr('data-url'),
      encodedUrl = encodeURIComponent(url),
      title = $this.attr('data-title'),
      tsina = $this.attr('data-tsina');
  var html = [
  '<a href="#" class="overlay" id="qrcode"></a>',
  '<div class="qrcode clearfix"><span>扫描二维码分享到微信朋友圈</span><a class="qrclose" href="#share"></a><strong>Loading...Please wait</strong><img id="qrcode-pic" data-src="http://s.jiathis.com/qrcode.php?url=' + encodedUrl + '"/></div>',
  '<a href="#textlogo" class="article-back-to-top" title="Top"></a>',
  '<a href="https://www.facebook.com/sharer.php?u=' + encodedUrl + '" class="article-share-facebook" target="_blank" title="Facebook"></a>',
  '<a href="#qrcode" class="article-share-qrcode" title="QRcode"></a>',
  '<a href="https://twitter.com/intent/tweet?url=' + encodedUrl + '" class="article-share-twitter" target="_blank" title="Twitter"></a>',
  '<a href="http://service.weibo.com/share/share.php?title='+title+'&url='+encodedUrl +'&ralateUid='+ tsina +'&searchPic=true&style=number' +'" class="article-share-weibo" target="_blank" title="Weibo"></a>',
  '<span title="Share to"></span>'
  ].join('');
  $this.append(html);
  $('.article-share-qrcode').click(function(){
    var imgSrc = $('#qrcode-pic').attr('data-src');
    $('#qrcode-pic').attr('src', imgSrc);
    $('#qrcode-pic').load(function(){
        $('.qrcode strong').text(' ');
    });
  });
});     
</script>









  </body>
</html>

